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Abstract 


'This  paper  describes  a  new  method  for  finding  nontrivial  solutions 

f\  x  >  **  *  <=> 

of  the  inequality  ^Ax  >  5),  where  A  is  an  mxn  matrix  of  rank  n. 

The  method  is  based  on  the  observation  that  a  certain  function  f 
has  a  unique  minimum  if  and  only  if  the  inequality  fails  to  have  a 
nontrivial  solution.  Moreover,  if  there  is  a  solution,  the  direction 
of  divergence  of  an  attempt  to  minimize  f  will  converge  to  a 
solution.  The  technique  can  also  be  used  to  solve  inhomogeneous 
inequalities  and  hence  linear  programming  problems,  although  no 
claims  are  made  about  competitiveness  with  existing  methods. 
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A  New  Method 


for  Solving  Linear 
Inequalities 
G.W.  Stewart 


1.  Introduction  j 

{ 

In  thifi  note  ve  outline  a  method  for  solving  linear  inequalities  of  the  I 

i 

form 


(1.1)  Ax  >  b,  ! 

where  A  Is  an  mxn  matrix  of  rank  n.  Since  any  linear  programming  j  ; 

problem  may  be  case  In  this  form,  the  method  Is  also  a  new  method  for  |  I 

linear  programming. 

The  heart  of  the  method  Is  a  technique  for  either  solving  the  j 

homogeneous  Inequality 

(1.2)  Ax  >  0,  x  4  0, 

or  determining  that  no  solution  exists.  The  underlying  idea  Is  simple.  ' 

Consider  the  function  \ 

T  H 

(1.3)  f(x)  •  1  exp  (.-Ax)  ; 

T  .1 

where  1  -  (1,1,..., 1)  and  for  any  vector  y  j 

exp(y)  -  (eyi,  ey2, . . . ,eym)  . 

Ue  shall  show  that  one  of  two  things  must  happen  if  f  i8  minimized  j 

iteratively.  If  (1.2)  has  no  solution,  then  f(x)  has  a  unique  minimum  j 

to  which  the  Iteration  must  converge.  If  (1.2)  has  a  solution,  then  the  :  j 


Iterates  will  grow  unboundedly  In  such  a  way  that  a  solution  can  be  computed 
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The  next  two  sections  of  this  note  will  be  devoted  to  the  homogeneous 
inequality.  In  the  last  section  we  shall  treat  the  general  linear  inequality 
(1.1). 


2.  Preliminaries 

T 

We  shall  use  the  following  notation.  Let  a^  denote  the  i-th  row  of 
A.  For  any  vector  x  set 

P(x)  »  (i:  a^x  >  0}, 

Z(x)  -  {i:  a^x  -  0}, 

N(x)  ■  { i:  a^x  <  0). 


If  x  is  a  solution  of  (1.2),  then  N(x)  *  0.  If  x^  and  x^  are 
solutions,  then  x^  +  x^  is  a  solution  and 

P(xx  +  x2)  *  P(x1)uP(x2), 

Z(xx  +  x2)  «  z(xx)n  z(x2). 

It  follows  that  if  there  exists  a  solution  to  (1.2),  then  there  exists  a 

„  *  * 
solution  x  for  which  the  cardinality  of  Z(x  )  is  minimal.  This  minimally 

£  it  it 

active  solution  need  not  be  unique,  but  the  sets  P  *  P(x  )  and  Z  *  Z(x  )  are. 

A  transformed  version  of  the  problem  will  be  needed  In  the  sequel.  Suppose 
that  (1.2)  has  a  solution  and  let  x*  be  a  minimally  active  solution. 

Without  loss  of  generality,  we  may  assume  that  the  rows  indexed  by  Z*  are 
the  last  rows  of  A;  l.e. 


where  .  AjX  >  0  and  A2x  •  0.  Since  x  i  0,  it  follows  that  A2  has  a 
nontrivial  null  space.  Let  V  •  (Vj  V2)  be  an  orthogonal  matrix  with  the 


I 


-3- 


coluans  of  spanning  the  null  space  of  •  If  we  set 

(2.1) 

and 


Bu  -  Vj 


“i  -  vi*’ 


then  the  Inequality  (1.2)  becomes 

(2.2) 


Bu  5 


BU  B12 


B 


22 


» 

U1 

. 

2 

>  0. 


Here  both  B^  and  B^  have  full  column  rank.  Moreover,  the  minimally 
*  * 

active  solution  u  corresponding  to  x  satisfies 


(2.3) 


Bu"l  ’  °> 


0. 


3.  Homogeneous  inequalities. 

We  begin  by  establishing  some  elementary  facts  about  the  function  f 
defined  by  (1.3).  Clearly  f  is  bounded  below  by  zero.  Its  gradient  and 
Hessian  are  given  by 

f  ’  (x)  -  -A?  exp  (-Ax) 

and 


where 


f"(x)  -  A  D(x)A, 


T  T  T 

D(x)  “  diag(ealx,  e8^31, . . .  ,e*®x) . 


Since  D(x)  is  positive  definite  and  A  is  of  full  column  rank,  f"  (x) 
is  positive  definite.  It  follows  that  f  is  strictly  convex  and  can  have  at 
most  one  local  minimum,  which,  when  it  exists,  is  also  a  global  minimum 
[2  ,  13.4.6,  §4.2.7].  Sufficient  conditions  for  the  existence  of  a  minimum 
are  contained  in  the  following  theorem. 
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Theorem  3.1  If  (1.2)  has  no  solution  then  f  has  a  minimum. 

Proof .  It  is  sufficient  to  show  that  f(x)  -*■  +  ®  as  |  |x|  |  * 

(any  norm)  [2,54.3.3].  For  any  x  with  | |x| |«  1  set 
$(x)  ■  min  (a^x:  ieN(x)}. 

Since  (1.2)  has  no  solution,  N(x)  is  nonempty  and  <|>(x)  <  0.  Clearly 
(£  (x)  is  continuous.  Hence 

0  *  sup  $(x)  <  0. 

lull- 1 

Now  for  any  x  4  0 

f(x)  >  l  e"aix  >  e~® I  I x II 
ieN(x) 

which  establishes  the  theorem. 

The  condition  in  Theorem  3.1  is  also  necessary.  However,  for  the  purpose 
of  this  note  we  must  take  a  more  detailed  look  at  the  properties  of  f  when 
(1.2)  has  a  solution.  This  is  most  conveniently  done  in  terms  of  the  trans¬ 
formed  system  (2.2)  and  the  associated  function 
g(u)  -  lTexp  (Bu) 

-  lTexp  (B11u1  +  B12u2)  +  JTexp  (B22u2) 

-  gj(u)  +  g2<u2>* 

Lemma  3.2.  The  system 

(3.1)  B22u2  >  0 

has  no  nontrivial  solution.  Hence  g2(u2)  has  a  unique  minimum 

(3.2)  Y  ■  «2(uP* 
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Ptpof.  Suppose  u2  le  a  nontrivial  solution  of  (3.1).  Because  B 
Is  of  full  rank,  ®22u2  **  0  •nd  henc«  B22u2  has  at  least  one  positive 

component.  From  (2.3)  It  follows  that  there  Is  an  o  >  0  such  that 

* 


0,ll“l  +  *12 u2  *  °- 


22 


Hence  the  vector 


ou. 


is  a  solution  of  (1.2;  with  Z(u)  a  proper  subset  of  Z(u*),  which 

.  * 

contradicts  the  minimality  of  Z(u  ) .  The  existence  of  a  unique  minimum  now 
follows  from  Theorem  3.1. 


Theorem  3.2.  The  function  g  satisfies 
(3.3)  g(u)  >  inf  g(v)  -  y 

where  y  is  defined  by  (3.2).  Moreover,  if  u^ 
g(u(k))  y  then 


(3.4) 


4k) 


U„ 


2.  Bjjttj^  >  0(-ln[g(u^k^) 


is  any  sequence  with 


?]> 


Proof:  For  any  vector  u  we  have 


(3.5) 

On  the  other  hand  if  we  define 


g(u)  *  Bj/u)  +  g2(u2)  >  g2(u2)  >  Y 


u  • 

a 


ou. 


Then 


11m  g(u i  )  -  11m  g.(au)  +  g2(u'J 
«-*•  a-***  * 


0  +  y  ■  y. 
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This  establishes  (3.3). 

(k)  (k) 

To  establish  (3.4),  let  u  be  any  sequence  with  g(u  )  -►  y. 

(k) 

Then  in  view  of  (3.5)  we  must  have  82^2  '  *  Y  which  implies  (3.4.1). 

(k)  (k) 

Now  since  we  must  have  g^(u  )  <  g(u  )“Y»  If  follows  that 

exp(-B12u2(k))  ©  exp(-B1;Lu£k))  <  g(u(k))  -  y 

where  ©  denotes  componentwise  multiplication.  Hence 

expC-B^u^)  <  [g(u(k))  -  y]exp(  B12u^kb 
or 

(3.6)  Bllul^k^  -  ”  ln[g(u^k^)  -  y]  -  B12u2^k^' 

(k) 

Since  u^  is  converging, (3. 6)  is  equivalent  to  (3.4.2). 

We  are  now  in  a  position  to  describe  a  method  for  solving  the  system  (1.2). 

(k) 

Let  be  a  diverging  sequence  of  radii  and  for  each  k  let  x  be  the 

solution  to 

(3.7)  minimize  f(x), 

subject  to  |l x II  2  < 

where  ||  *  ||  ^  denotes  the  usual  Euclidean  norm.  The  convexity  of  f  and  the 

(k) 

constraint  insure  that  x  is  uniquely  defined. 

Now  if  (1.2)  has  no  solution,  theorem  3.1  assures  us  that  for  some  finite 
(k) 

k  the  solution  x  will  lie  in  the  interior  of  the  constraint. 

On  the  other  hand,  if  (1.2)  has  a  solution,  each  x^  will  lie  on  the 

boundary  of  the  constraint  (i.e.  ||x^||  2  -  p. ) .  Moreover,  from  (3.4)  it 

(k)  * 

follows  that  the  components  of  Ax  corresponding  to  p  will  diverge  to 
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*  if 

4  «  while  the  components  corresponding  to  N  will  converge.  Once  N  has 


been  recongnized,  we  can  compute  the  transformation  V  to  the  u -coordinate 

system  [cf.(2.1)].  For  each  k  we  then  compute  trial  solutions  of  the  form 

-(k>  „  ,,T  (k)  „  (1) 

x  ■  VjVjX  ■  V^u^ 

(V\  _(ip\ 

(l.e.  the  vectors  obtained  by  setting  u  “0  so  that  A^x  '  ■  0) .  It  is 

possible  that  initially  the  x  may  not  be  solutions ,  owing  to  the  supression 
(k)  -(k) 

of  the  terms  B^2U2  *n  ^1X  *  ^Ut  (3.4.3)  insures  that  ultimately 


BUU2 


(ir\  -(V)  -(V) 

v  9  *  A^xv  *>  0,  and  at  that  point  xv  7  is  a  solution. 


There  are  a  number  of  observations  to  be  made  about  this  method. 

Remark  1.  The  method  produces  a  minimally  active  solution.  This  will 
be  important  in  the  next  section. 

Remark  2.  We  have  not  specified  any  particular  algorithm  for  solving 
the  constrained  problem  (3.7).  However,  we  note  that  it  is  about  as  nice  a 
problem  as  one  could  wish  for.  Within  the  constraint  f  is  uniformly  convex, 
and  its  first  and  second  derivatives  are  easily  computed.  The  constraint  is 
not  only  convex,  but  the  projection  onto  its  boundary  is  trivially  computed. 
Finally,  in  the  passage  from  to  the  vector 

makes  a  natural  starting  point  from  which  to  find  x^+1\ 

Remark  3.  Strictly  speaking  the  method  is  not  an  iterative  method,  but 
a  finitely  terminating  method  with  an  inner  calculation —  the  solution  of  (3.7)  - 
that  will  almost  certainly  be  done  by  iterative  techniques.  A  true  iteration 
may  be  obtained  by  observing  that  the  vectors  x^/  ||x^j|  converge  to 
a  solution  of  (1.2).  However,  the  solution  thus  obtained  may  not  be  minimally 

(V) 

active,  since  there  is  nothing  to  prevent  the  components  of  AjX  from 


diverging  at  different  rates.  The  author  conjectures  that  this  will  not 

00 

happen  if  for  the  sequence  x  generated  by  (.3.7)  . 

Remark  4.  A  possible  alternative  is  to  turn  an  unconstrained  minimization 
method  loose  on  f  and  see  what  happens.  This  will  clearly  be  satisfactory 
when  (1.2)  has  no  solution,  since  most  well  constructed  minimization  methods 
are  globally  convergent  when  applied  to  convex  functions.  Unfortunately 
for  the  case  where  (1.2)  has  a  solution,  the  divergence  properties  of 
minimization  methods  have  not  been  well  studied.  Not  only  may  the  iteration 
produce  a  solution  that  is  not  minimally  active,  but  one  must  deal  with  the 
possibility  that  it  may  produce  values  uniformly  greater  than  the  infimum  y* 
Nonetheless,  the  simplicity  of  the  resulting  class  of  methods  makes  the  analysis 
well  worth  undertaking. 

Historical  note.  The  method  described  in  thi6  note  has  its  origins  in 
the  maximum  likelihood  analysis  of  log-linear  models.  Specifically,  one 
is  given  a  vector  n  of  independent  Poisson  random  variables  with 
E(n)  «  exp(Ub),  where  U  is  a  matrix,  often  of  formidable  size.  Up  to  a 
constant,  the  log-likelihood  function  for  any  estimate  of  E(n)  is 
(3.8)  £(n,  b)  «  n^Ub  -  l^exp(Ub), 

which  is  seen  to  be  an  elaboration  of  our  f.  When  some  of  the  elements  of 
n  are  zero,  the  maximum  likelihood  estimate  of  the  corresponding  means  may  also 
be  zero,  in  which  case  an  iteration  for  maximizing  (3.8)  must  diverge,  since 
Ub  estimates  the  logarithm  of  the  means. 

Haberman  [1,  Theorem  2.3]  has  given  a  necessary  and  sufficient  geometric 
condition  for  the  existence  of  a  maximum  of  £(n,b).  In  the  course  of  work  on 
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the  numerical  analysis  of  log-linear  models,  the  author  reduced  this  conditon 
to  the  nonexistence  of  a  solution  of  (1.2),  where  A  will  generally  have 
dimensions  much  smaller  than  U.  It  was  then  an  easy  matter  to  recognize  that 
the  inequality  corresponded  to  a  trivial  log-linear  model  with  a  log- likelihood 
function  that  is  essentially  -f.  Thus  the  existence  or  nonexistence  of  a 
solution  of  (1.2)  is  equivalent  to  the  nonexistence  or  existence  of  a  minimum 
of  f . 

4.  Coda  —  inhomogeneous  inequalities* 

It  is  easy  to  see  that  the  inequality  (1.1)  is  equivalent  to  the 
existence  of  a  solution  of  the  inequality 


>  0 


(4,1)  [  A  -  b  y 

0  1  J  n 

with  r\  >  0,  in  which  case  x  -  y/n.  Since  the  method  of 
minimally  active  solutions,  when  applied  to  (4.1)  it  will 
solution  with  n  >  o,  or  it  will  produce  one  with  n  «0 
solution,  in  which  case  (1.1)  has  no  solution. 


§3  produces 
either  produce  a 
or  indicate  no 
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